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ABSTRACT 

A test of the hypothesis that the second parameters 
("success" probabilities) of a number of paired binomial 
distributions are pairwise equal is derived under weak 
assumptions 0 Computer codes necessary to implement the 
procedure are given and a case study is used to demonstrate 
the procedure 0 Some other procedures for testing the same 
hypothesis under stronger assumptions are discussed and 
compared with the given procedure „ A rapid approximate 
procedure is also given* 
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CHAPTER I 



INTRODUCTION 

In certain experimental situations one is interested 
in the hypothesis of pairwise equality of means in (say) 
n pairs of binomial distributions, but has no interest in 
what that mean (for a given pair) is. Specifically, if we 
denote the "success probability" parameter of the first 
member of the i tlri pair by d^ and the same parameter of the 
second member by f^, then it is desired to test the 
hypothesis 

Hq i = ^ ^ • ••» n 

against various alternate hypotheses which are left un- 
specified at this point. Note that the relationships (if 

any) between d. and d. or between f^ and f for i ^ j, are 
i j i j 

of no concern, nor are the numerical values of d, and f . 

i i 

A practical example of the need for this type hypo- 
thesis test arises in the examination of a detection system. 
A basic requirement of a detection system is that it should 
perform better than a "random" system, which is defined as 
one in which d, the probability of detecting a target 
(given that one is actually present) is equal to f, the 
probability of giving a false alarm. A system may be con- 
sidered better than random if d > f . 
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Suppose we have a system which may respond, differently 
under different operators. If d and f^ are the detection 
and false alarm probabilities* respectively, for the i^* 1 
operator, we are naturally interested in testing 



H ; d. 

o i 



f 



i; i = 1.2 



g o o o 



, n against, say 



V d, > f. 



i; i = 1,2 



9 o o o $ 



n e 



In some situations, one might require a two sided test or 
perhaps a less stringent alternate hypothesis. 

In any case, the matter of interest is whether the 
system performs better than "random" for each operator, 
without regard to possible differences between operators. 

For the remainder of this paper, the detection system 
example will serve as a "prototype" case. It is hoped that 
this will result in Increased intuitive appeal and clarity 
of presentation, 

A typical experiment conducted to facilitate per- 
formance of the desired hypothesis test involves n operators, 
th 

the i of whom makes r, detection attempts with targets 

actually present and s t attempts with no targets present. 

The data then consists of n pairs of observations (j^,k^) 

where j. is a realization of the random variable J , the 
i i 

th 

number of true detections made by the i operator from 
among the r^ attempts with targets present, and k^ is a 
realization of the random variable , the number of false 
alarms registered from among the s^ attempts made without 
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targets present. We assume the i th operator makes true 
detections with constant probability d^, and generates 
false alarms with constant probability f i< Thus J is 
binomially distributed with "success" probability parameter 
d^ and number of trials parameter r^ (hereafter written 

j j 

Ji = b(r 1 ,d i )). Similarly, we have = b(s 1 ,f i ). 

We will assume that the experimental design and 
general conditions are such that the following assumptions 
are satisfied; 

1. r^ ^s^m VI (made only for mathematical simplicity) 

2. J £ b(m,d i ), K 1 £ b(m,f i ) Vi; i = l,2, n 

3. and K i are independent V 1 ? 1=1,2, ..., n 

4. and are independent for i V J 

5. K and K. are independent for i V 3 

-L J 

SOME EARLIER APPROACHES 

Before proceeding with our development of the desired 
hypothesis test, we briefly discuss two procedures which 
have been used in the past. Since both of these procedures 
require assumptions stronger than those listed above, they 
are of restricted applicability. 

The Equal Performance Assumption . If the additional assump- 
tion that d = d Vi and f = f Vi, then the experimental 
i i 

data may be summed into a single pair of observations 
n n 

j' = £ j. and k' = 2 kj, where j' and k* are, respectively, 

i=l 1 i=l . 

realizations of J* and K* , where J* = b(nm,d) and 

K’ = b(nm,f). 
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The natural hypotheses under test are then 



H q ; d = f against s d>f 



which may readily be tested by the exact method discussed 



A hypothetical example serves to Illustrate the 
dangers of improper application of the foregoing procedure. 
Suppose that n/2 of the observed pairs were (m,0), while 
the other n/2 were (0»m)„ Then j ' = nm/2 and k* = nm/2 
and the hypothesis that d = f would be accepted. What has 
probably happened 9 however is that d^ > f for about half 

the operators while d. < f . for the remainder. Summing 

i i 

over all individuals has obscured information to this 
effect contained in the data. 

In an extreme ease 9 such as the above, the experi- 
menter should recognize that the equal performance assump- 
tion does not hold 0 However suppose the case were less 
extreme,, say that given in Table 1 (assume m>6). 



1 ? 
below or by an appropriate asymtotic test 



TABLE 1 



Results of a Hypothetical Experiment 



Number of observations 



Value of observation (J^k ) 



n/4 

n/4 

n/4 

n/4 



(m - 2, 4) 
(m - 6, 2) 
(5, m - 3) 
(1, m - 5) 
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Here, testing under the assumption of equal performance 
yields j * = (n - l)m/2 and k* = (n - 1 )m/2 , and again, one 
would accept the hypothesis that d = f. ^ith this result, 
the experimenter would not be likely to suspect the equal 
performance assumption, yet it is not clear that it is 
satisfied. 

One might attempt to "validate" the assumption of equal 

r> j, 

performance by conducting a homogeneity -' 9 test on the ob- 
servation vectors (J-pjg J n ) and ( k i’ k 2 or 

goodness of fit tests-5 to some binomial distributions. 
However, the homogeneity test for binomial data is a strictly 
intuitive procedure, and thus not entirely desireable; also 
the selection of which binomial distribution to test the 
data against poses a considerable problem. Further, in both 
cases, the size of the overall procedure is difficult to 
determine. ^Je therefore conclude this discussion by noting 
that testing for homogeneity among binomial samples is an 

area in which additional work appears to be needed. 

£ 

The Analysis of Variance Approach . Bartlett has suggested 
and Curtiss”' 7 has shown that the arcsine transformation 
Y = arcsin where X = b(m,p), imparts to Y an approximate 

normal distribution with mean arcsin 7 p~ and variance l/4m. 
(Denoted Y £ N(arcsin Using this technique in ■ 

the prototype case we have = arcsin Jj^/m and = arcsin 
/K^/m distributed approximately normal with means arcsin 
y~d~ and arcsin respectively, and the same variance l/4m. 

If we now think of having a target present as treatment D 
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and having no target present as treatment F, the J* repre- 
sents a "response" under treatment D, while represents a 
response under treatment F. We can now model these responses, 
say as J» = ^ t C p£ . where 

- mean (transformed) response under treatment D » 

= differential (transformed) response under treat- 
ment D for the l u operator 
= (transformed) sampling error for the observation 

4** ly-j 

on the i operator under treatment D. 

K* could be modeled analogously as 7* * 4m; + £ *; and we can 
assume that: 

lo <^. £ - is asymtotlcally N(0 ) y = K,D; i = l,2,...,n 

2 0 € w is asymtotlcally N(0 , <r 2 ) -v = k, D; i = l,2,...,n. 

The random factor model , two way classification Analysis of 
Variance may then be used to test 

H_ ; d, = f. against the somewhat restricted and 

0 1 i 

unnatural alternate hypothesis 

L: d, 4 f for some i„ 

1 i i 

A further problem with this approach is that it requires 

m "large" and d, and f not "extreme", say 0.2 <d. <0.8 and 

lx 1 

0.2<f J( <0o8 in order for the arcsine transformation to 
yield reasonably normally distributed random variables. 

Since we cannot (indeed, may not want to) assure that these 
additional requirements are met, this approach is of limited 
applicability. 
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A MORE GENERAL APPROACH 



The purpose of this paper is to develop a procedure 
to test the hypothesis H Q : d^ = f Vi against various 
alternate hypotheses (as yet unspecified) under the four 
assumptions listed above, and no others. 

In Chapter 2, we develop a graphical representation of 
the observations (j^,k ), derive the likelihood ratio test 
critical region characterization, present an exact test for 
Hq : d^ = f against : d i ^ (or d i > f 1 ) for a given 
individual, and explore three apparently unfeasible 
approaches. 

Development of a general two stage test procedure is 
given in Chapter 3> and in Chapter 4 we adapt the two stage 
test procedure to the prototype situation, present a de- 
tailed case study, and make some comparisons with the earlier 
approaches. We conclude with the derivation of an approxi- 
mate, but rapid procedure which may be useful in some cases. 
Computer codes necessary to implement the two stage procedure 
are given in two appendices. 
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CHAPTER II 



BACKGROUND AND "CONVENTIONAL" APPROACHES 

The user of "conventional" approaches to testing the 
hypothesis Hq : d, = f against some alternate meets with 

difficulty due to the need to specify a value of d^ (or f ) 
as will be shown below. Before doing so, however, it is 
convenient to introduce a graphical approach to the problem, 
derive the likelihood ratio test, and consider a special 
case (n = 1 ) . 

A GRAPHICAL REPRESENTATION 

For ease of communication, it seems advantageous to 
utilize a graphical display of the data such as shown in 
Figure 1, where the observations (j k, ) are normalized (by 

X 1 

dividing by m) and plotted in a unit square. Intuitively, 

one feels that if d. =• f. Vi, then the points (i /m,k./m) 

i i II 

should all. lie near the line j = k , whereas if d > f Vi 

1 XI 

then the points (j^/m,k^/m) should tend to lie to the 

right of the line = f^, etc. In a sense, the test of 

H q ; d_ = f. Vi against some alternate hypothesis is a test 

of "closeness" of the points (J /m s k /m) to the line j. = k . 

II ■'■I 

This concept will be helpful in the succeeding discussions, 

THE LIKELIHOOD RATIO TEST 

It would be desireable to have a likelihood ratio 
test, since the Neyman-* Pearson Lemma then provides that for 
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simple hypotheses, the test is best of Its size. Let us 

consider specifically the test of H^: = f^ i against 

H 1 : d 1 ^ i. Let v be the vector ( ,k 2 , . . . , J n , ) . 

By independence the likelihood of v is given by 

n fm m Ji k < m-J* m-k \ 

L(v) = iHL 1 ( Ji )(k i )(dl) ( V (1 "V j 

The parameter space <n>is given by 




(d 1 ,f 1 ); o < d l 



< l, 



0 < f ^ < 1 , 1=1,2 




(2) 
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while the null space CJ is given by 



CO = | (d 1 ,f i ); 0< <^<1, 0<f i <l, d 1 =f' 1 » i = 1 ,2 , . . . ,n 

( 3 ) 

It is convenient to operate on Ln L rather than L directly 
so we maximize 



n 

LnL = 2 

i=i 



■j In d. + kln f, + 
i i i i 



(m-^ )ln(l-d ) + (m-k )ln(l 



( 4 ) 



over the null and parameter spaces , Adopting the convention 
0**Ln 0= 0, (that is extending the function x < ln x to 0 by 
continuity), and maximizing yields 



max 

u> 



LnL(v) 



n f “l* 

= i-l kl ^ + ^ 2m "^i“ K i ^ ln ^ 1 v 2m ^ 

( 5 ) 



(J i 1 n ( T ) ' l ‘ k i ln( ir H(a “ j i ) 1 n ( 1 “T' ) + 

k I * 

(m-k )ln(l - -^) f . (6) 

i m j 

The logarithm of the likelihood ratio (v) = ;i ^j X L(v )/^j Q X L(v ) 
is given by 



and 

max. T x ^ - 
n LnL(v) — 2 

^ u 1=1 



* up to additive constant terms 
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) + (2m- < 3 i “k i )ln(l- 



Ln A(v) = SI (j 1 + k l )ln(- 25 - 

U k i 

- j.ln (-±) - k ln(— ) - (m-j 
i m 1 m i 



)ln(l - — ) - (m-k 
m l 




and. we may characterize the test by rejecting when Ln X(v)<C 
(constant). The boundary of the critical region is then the 
hypersurface in (2n+l) dimensions defined by Ln X(v) = C, 
which is difficult to visualize. However, by the indepen- 
dence among pairs of observations, contour lines of Ln X(v) 
in any of the n planes defined by j^ = 0, k^ = 0, i^l for 
1 = l,2,...,n, will be identical, and as shown in Figure 2. 
It may be noted that Ln X(v) could be quickly evaluated by 
plotting the points (j^/m,k. /m), i=l,2,...,n on Figure 2, 
and summing the values of the contour lines on which they 
fall. This reinforces our original intuitive notion of 
rejecting Hq if too many points fall "far" from the diagonal 




For a one sided test, say : d i > f ^ \/i, the likeli- 
hood contour line plot corresponding to Figure 2 is shown on 
Figure 3. 

It appears we are now ready to complete the procedure. 
We need only find C such that P^Ln M"v) < C] = 0( xtfhere o(is the 



* up to additive constant terms 



1 ? 




Contour lines of Ln (v) In the plane 
defined by J^=0, k^=0, i^l(two sided test) 

desired siae of the test. But here is where the difficulty 
'.-.rises. The conventional approach to finding P|_Ln >(V)<3j 
oald be to note that 

•In X(v) = C v ] = L ?[V = v ] where (8) 

SCly. S 

I v - £s; Ln > (v^ ) = C-r^j , and so if ; C y < C^j we have 

riU- MV) <C] = L L P[V - v s J. ( 9 ) 

-r<r c Sil -r 
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1.0 






o.o 

0.0 j ^/m 1 . 0 

Figure 3 

Contour lines of Ln (v) in the plane 
defined by ^ = 0, 1^ = 0, i^l (one sided test) 

Both P and 1^ are readily found for all C and , but the 
c 

probability that V = v cannot be specified, for assumption 

s 

£ (i 

2 states only that J^= b(m,d^) and = b(m,f^) with d^ and 
f^ unspecified. This inability to complete the relationship 
between the size of the procedure and the critical value C 
renders the "conventional" approach useless. 

SOME "DIRECT" ATTEMPTS TO FORMULATE TESTS 

Three comparatively straightforward attempts to circum- 
vent the lack of specific distributions for and were 
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made. A brief summary of each with an explanation of why it 
failed follov;s. 



The first attempt involved the transformation 



a = (— - —)J\ , which is a projection of the points 
i v , n m 




Figure 4 



The Angular Transformation u( J^/mjk^/m) 
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illustrated in Figure 4. For example, the point A: (1/2, 3/4) 
is transformed into u( 1/2, 3/4) = Intuitively, one 

would hope to find a good procedure which rejects if too 
many of the u^ fall close to the end points of the interval 
( - 71/2 , 7l/2 ). The details of why such a procedure is not 
good are burdensome, but the cause is easily shown graphic- 
ally by the points B and D in Figure 4. Note that both B 
and D map to the same point E on the U axis, yet B represents 
a Ln X value of - 1, while D falls on the Ln X = - 4 contour. 
Thus if C, the critical value, was - 2, say, point B should 
cause acceptance of , while point D should cause rejection, 
yet given only point E on the U axis, we cannot determine 
whether it is the image of B or D, hence this approach 
fails. (More complicated transformations along the same 

general lines were also attempted, but without success. ) 

n 



The second attempt involved use of the sums 2 j< and 
n i=l 1 

2 k. as test statistics. Again, the details of the break- 
i=l 1 

down are burdensome, and a graphical argument demonstrates 

the problem quite clearly. As an example, we shall consider 

two different experiments, each with n=2, with results 

plotted on Figure 5» The points A and B are from the first 

experiment, the points A’ and B* from the second. The sums 
n n 

2 j. and 2 k, are the same for each experiment (1.1 in 
i=l 1 i=l 1 

all cases), yet clearly, the first experiment (points A and B) 

should tend more toward rejection than the second (points A’ 

n n 

and B'). In short, 2 j. and 2 k are not sufficient 

i=l 1 i=l 1 

statistics for d^ and f ^ , i = 1 ,2 , , . . ,n. 
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Figure 5 

Two divergent experiments with equal data sums 



Discussion of the third "direct" attempt is best 
deferred until the end of the next section, which establishes 
some necessary background and notation. 

A SPECIAL CAST (n = 1 ) 

•Then only one operator is used in the experiment, the 
procedure known as Fisher’s Exact Test for percentages is 
applicable . A brief derivation follows. The test of 
interest is now 

3 : d^ = f^ against (say) : d^ >f^. 
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Now under Hq, and by independence 



P[J 1 = = p[j.,=q] . pC'Kj-^] 



m!m! d^ 



^ 1 + k l 



(1 -di ) 



jj_ J (m-Jj. ) Ik! (m-k^ ) ! 



( 10 ) 



But P[ J i = ji» Kj = k^ J is equivalent to J + = j^+k ]. 

At this point it is convenient to introduce the random 
variable = J 1 + so that we have f jr^ = j ^ 3 equivalent 

to p[j^ = ,) ,L = lj 3 where 1^ = J ^ + k^ . Equation 10 may now be 



written 



2m - 1. 



m !m Jd^ ' (1 - d^ ) 

P[ J 1 = = \1 = j i j(i i - ^JKm- j^Km+^-l^! 



( 11 ) 



and we note that we must have 0< <m, O^j., <l^<2m. Now 

(still under H_), the random variable L« is the sum of two 
0 •*- 

independent identically distributed binomial random variables 
and so is binomially distributed itself. Specifically, 



= b(2m ,dj ) , so 



p[l 3 = 1^ = 



1h 2m - 1 1 

(2m)! d 1 ( 1 - a 1 ) 1 

(2m — 1 1 ) I 1^ I 



( 12 ) 



By the definition of conditional probabilities 

pl-J < r P ^ J 1 = J 1» L 1 =1 1^ 

P[ J 1 = Jj L 1 = l x ] = " p[ L ‘ 



( 13 ) 



25 



Substituting (11) into (12) and (13) yields 



p C J i = JiK = ii] 



1. 2m - 1 1 

m !m id^ (1 - d^ ) 

~1 ! t 1 ! - Ji)!(m - j 1 )!(m-k-j 1 -l 1 )i 

I. 2m - 1, 

(2m) Id^ 1 (1 -d x ) 



(2m -1^! 1^ ! 



m !m ! (2m - 1^ ) ! 1^ ! 

(2m) 5^ ! (l j - ^ ) ! (m - ^ ) ! (m + ^ - ^ ) ! 



(1*0 



Rearranging appropriately yields 



p E J i = J i 




(“) 
J 1 



m 



2m 

1 



(15) 



which is the hypergeometric distribution with first parameter 
(population size) 2m, second parameter (sample size) 1^ , and 
third parameter (proportion) 1/2. 

■Je now have the distribution of the number of true 
detections J-^ given 1^ total indicated detections, and most 
importantly, it is independent of d^ and f^. Let us now 
develop a likelihood ratio test of HqJ d^ = f^ against some 
as yet unspecified alternate hypothesis. Letting Mj-Jl^) 
denote “Jf L( t ) 1 jl 1 ) where cv = ^*[0,1]; ^ = fj, 

<Hi» = ^ d x ; 0 < d 1 < and L(JjJl^) is the conditional likelihood 

of given 1^, we have 




p C J i = J i| L i = 1 i’ H i ] 



( 16 ) 
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Now under Hi 



p C J l = 3l' L l = 1 ll H l> 



and 



mlmjdj ' f^ 



1 1~^1 



m-Ji m+j -1 
(l-d x ) 1 (1-f ! ) 1 X 



! ) ! - 1 1 ) ! 



(17) 



P[Li = l x 



n m ~ 1,-a m-li+a-i 

H 1 (^l) f l 1 U-fj) 1 

1 a=a^ a! (m-a)l (ljL-a) I (m-lj^+a)! J 



(18) 



where a = max t • *.-} and b = min (- .u ■ 
Substituting (17) and (18) into (13) we have 

1/ { Ji I ! (m-Ji ) ! (m-l 1 +J 1 ) 






b 

v 


\U-fJ 


ct-3l 


I 


Lh'd-dll 





a=a a ! (m-a ) ! (1^ -a ) ! (m-l^-t-a ) ! 
And substituting (14) and (19) into (lS), we obtain 



(19) 



X(J x li) = 



m !m ! (2m-!] ) !l^ ! 

(2mTi 



'pid-fi 



a=a 



a-j. 



c a ! (m-a ) ! (1^-a ) ! (m-l^-i-a ) 



( 20 ) 



Let us now consider a specific alternate hypothesis, say 

di (1-f-i ) , 

then have — — — - >1 and X( j 1 1., ) is mono- 

f„ M -d- } 1 1 1 



H l’ d l >f l- We 



f l<l" d l> 

tonically decreasing in j^, Since we would reject Hq if the 
observed test statistic X(J^jl^) is sufficiently small, say 
less than some critical value C, we can perform an equivalent 
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test by rejecting Hq if the observed value of j^ is greater 
than C* , where j^C' is equivalent to X(j ] |l^)<C. He 
note also that this test is uniformly most powerful for 
since the procedure would be the same for all d^ >f^. 

The size of the procedure is >C'|i^,Hq], which can be 

readily computed using equation 15, and the power at a 
specific point (d^,f^) can be computed using equation 19 . 

For a two sided test, i.e. : d^ ^ f ^ , a similar 
development shows that the optimum rejection region is 
characterized by the rule "reject Hq when j-j <A or j^ >m-A" 
where A is such that 



P[J 3 < A 



l 1 ,H 0 ] + P[J 1 >m-A|l 1 ,H 0 ] = 2p[J 1 <A|l 1 ,H 0 ] = a 



(Of course, not all values of ct€[0,l] are available. We 
assume that a is chosen from among the realizable values. 
Randomization to achieve arbitrary sizes may be used but we 
shall omit the details. To include such a consideration here 
would only add complication without adding substantive 
information. ) 

It now seems natural to extend the above result to 

cases where n>l. This was the third "direct" attempt 

alluded to in the previous section. Specifically, it was 

hoped that the families of probability functions 
n n n n 

P[ 2 J, I £ 1, ,H_] and p[ E J, 2 1. ,E. J could be obtained 

i=l 1 I i=l 1 0 i=l 1 ' i=l 1 1 

and used in a fashion analogous to the preceeding. However, 

whereas the function P[Jj jlj ,HqJ was independent of d^ and 
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n n 

f\ , such is not the case for pf 2 J. £ 1 ,H 1. The need 

1 1=1 1 1=1 i ° 

to specify d i and f^, i=l,2,...,n, which is not per- 

missible under our null hypothesis, forced the termination 
of this approach. 
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CHAPTER III 



THE GENERAL TWO STAGE TEST 

While directed at the prototype experiment, the follow- 
ing discussion has somewhat broader applicability than to 
only the specific situation discussed heretofore in this 
paper. For that reason, we will adopt a slightly more 
general notation for this chapter only. We will consider a 
test of the hypotheses 



V 


<D 

H* 

II 


hi 


V 


hi - 


®di 



Vi against 

I = A. Vi 
1 



and suppose that we have available tests T^ thru T n of the 
respective hypotheses 



H Oi : . 0 fi = 9 di a£ainst 



:i li : 9 fi “ 9 di 



= 



for each i: i=l,2,...,n. We shall denote the size and 
power of T^ by ou and , respectively. 

It seems reasonable that the outcomes of each of the 



tests T. thru T should contain some information about 

and . We will develop a procedure by which the results 

of T^ thru T n (hereafter called the first stage tests) can 

be tested in order to make inferences about and . 

This test on the results of T-, thru T will be called the 

1 n 
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second stage test, and the complete procedure (applying the 
first and second stage tests) will be called a two stage 
test. tie begin with a special case before developing the 
general procedure. 



IDENTICAL FIRST STAGE TESTS 

Suppose that is of the form : | | =A Vi. 

Then the respective alternate hypotheses for the first stage 



tests will be H 



li* 



© 



fi 



- ©^. I = A for each test T^ , 



i=l,2,...,n. Now if T^ thru T n are identical, that is 
have the same operating characteristic curve, and we apply 
each at the same size (a^ = a Vi)» they each provide the 
same power (tt^(a) = rr(A)Vi). Now let us define the 
random variables X^ ,X 2 , . ° . ,X n by 



h - 



fo if T^ results in acceptance of H Q ^ 
1 if T^ results in rejection of H Qi 



n 



Under H n all of the H. are true, so if Y = 2 X. then 

d 01 i=l 1 

Y = b(n,a). Similarly if is true, then all of the 

are true and here Y ^ b(n,n(A))» Let us denote the (general) 

distribution of Y as b(n,p). Then if we test 



HqI p = a against H| : p = n(A) 

we have actually tested Hq against since is true if 

and only if the EL. are true for all i and H 0 is also true 

Oi 0 

if and only if all the are true. In similar fashion 

implies and conversely. 
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Using the well known likelihood ratio test of 
Hq: p = 0( against H* : p =7 r(A), the procedure will be to 
reject H Q if the observed value of Y exceeds a critical 
value y* where 

p[Y y* | H*] = 2 # (g)a k (l-a) n k = a’ (1) 

u k = [y ' ] +1 

— # 

and the symbol j_-] denotes the greatest integer function. 
The test of Hq against H| is, of course, the second stage 
test. Now since we can substitute Hq for Hq in the left 
hand expression of equation (1), we see that a’ is the size 
of the two stage procedure. Likewise, the power of the 
second stage test is given by 

P[Y>y’ | HI] = S, (g) n(A) k (l -TT(A)) n * k =TT- (2) 

1 k=[y] +1 

and here too we can replace Hj by H^, so n' is the power of 
the two stage procedure. It is interesting that a’ (the 
size of the two stage test) may be set independently of a, 
the common size of the first stage tests. 

An examination of equation 2 suggests that the nota- 
tion tt' = tt'(A, a*, tt(A»oO) should be used to emphasize 
the functional dependencies involved. We will write the 
equivalent form tt* = rr*(A, a', a). Thus, while a* may be 
set independently of a, the choice of both a and a* in- 
fluences tt'. Generally speaking, we will be given (or be 
willing to specify) A and p( * so our concern would be to 
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choose an a that maximizes tt’(A, a', a). This requires 
solution of the program 



Maximize rr* = 



K=[y j ti 




Subject to; 




a* , 0 < a < 1 . 



If the appropriate OC Curves for the second stage test are 
available (that is if sufficiently extensive binomial tables 
are available), a simple search procedure may be used to 
find the solution. Otherwise it appears that some iterative 
technique will be required. In any case we will not consider 
this problem further in this paper, except as it applies to 
the prototype case (Chapter 4 ). 

Two further observations are appropriate. First, the 
two stage test may be one sided if one sided first stage 
tests are available. Similarly, either or both of and 
can be made composite provided the corresponding 
and/or can be made composite and with a slight modifi- 
cation in the definition of power. The derivation in such 
cases requires only minor modification of the foregoing. 
Second, there is no assurance that the two stage test repre- 
sents a uniformly most powerful test, even when the first 
and second stage tests are in themselves uniformly most 
powerful. It appears that, in general, the two stage test 
may not be most powerful, although its power characteristics 
seem to be good. „7e return to this question in Chapter 4 . 
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GENERAL FIRST STAGE TESTS 



VJe now relax the requirement that the first stage 
tests Tj jTg* . • • jl^be Identical and assume only that 0^ and 
77^ (^) are specified for each T^ , i = l,2,...,n. Ne will con- 
sider testing the hypothesis 

H 0 1 ®fi = 9 dl Vl a £ alnst = A i Vi 

for which the appropriate first stage tests are, for each i, 

H oi : ®fi = ®dl asainst H li ! l®fi-®di I - 



If some of the first stage tests are identical, we will 

group them together, so we will have (say) k groups of 

identical tests (that is, identical within a group) to 

consider. (For example, if no two tests T. , T.; i^j are 

1 J 

identical, then k=n, whereas if all tests are identical 

k=l.) Let 0^ denote the number of (identical) tests in 
th 

the j group, j = l,2,...,k, and relabel the ot^'s and tt, ' s 

so that a and tt represent the common size and power, 

J j 

respectively, of the 0. tests in group j. Now using argu- 

J 

ments identical to those in the previous section, we have as 
an equivalent test of against , the second stage test 



H* • 

ii 0 • 



p = a against HJ : p rt = tt, 

0 1 0 



Here p^ denotes the k component vector of success prob- 
ability parameters in the joint distribution of 

Y = (Y , Y_,...,Y ) where Y. is the number of rejections 
12k 0 



3 ^ 



(of ) from the tests in group j. (Thus Y^c jo ,1 , . . . ,0 *j . ) 

The vectors a and tt denote (a. ,a 0 ,...,a ) and (tt .tt , ...,tt ), 

1 c k 12 k 

respectively. Now if a’ is the size of the second stage 
test, we have P [[rejection of Hq | HqJ = P Rejection of 
H i| H o 1 so a' is the size of the two stage procedure as 
well. Similarly, the power of the second stage test tt', 
is P [[rejection of Hq | Hq } which is equivalent to P [ 
rejection of Hq j , so n' is the power of the two stage 
procedure also. 

Let us now get the second stage procedure. By inde- 
pendence 



k 

P[Y= y] = IT P[Y< =y .] . 

j=l J J 



(3) 



Under H* , Y is the number of "successes" in 0 trials at 
° j J 

constant probability a., or Y. = b(0.,a ), j •= l,2,...,k. 

J J d J J 

Similarly under H* , we have Y. = b(0 ,tt ), j = l,2,...,k. 

1 J J j 

Then, by equation 3> under Hq 



l (y) = p[i=y I H’] = 



k f0i 

■ £ fe 1 



y 



j (1 - a,)' 



0j -y 



'} 



J f (4) 



and under H* 
1 



L. (y) = p[y= y I H»] = 7 r K: 
1 1 j=l V- 






■} 



( 5 ) 
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Thus the likelihood ratio >\(y ) = L c (y )/ (y ) is given by 



K 

X(y) = c • 7 T 
3=1 



OjU-ii,) 



.^1 



n . (1 - a . ) 
■ J J . 



k 



where the constant C is 77 * 

3=1 

have rr. >a Vi, hence, 

J “ 3 



1 - a 



1 - TT 



3** 



0 • 



( 6 ) 



In general we will 



a . (1 - n . ) 

— — — < 1 V3» so \(y ) is monotonically increasing 

n . (1 - a . ) ~ 

J J 

as a function of any component y. of y. Characterization 

J 

of the optimum rejection region is best accomplished by 
considering 



Ln >v(7) =2 y . flna, - ln(l - a, ) + ln(l - tr. ) - In n .] + InC. (7 ) 
J J J J J 



He will reject Hq when ln Xy ) is less than some critical 

value, say_j\_. Since the quantity in the brackets £ in 

equation 7 is negative for all j, the general tendency will 

be to reject Hq when several or all of the y^’s are Si ^.Wu ei 

(near 0.), an intuitively appealing result. 

J 

Specifically, we may label all possible outcome vectors 

~y so that ^ (y^ ) < X(f i+ 1 ) » i = 1 ,2 , . . . ,v-l , where v is the 

k 

number of different y's possible. (v = 2 [0. + 1].) Then 

J 



3=1 



the critical region consists of the set of vectors 

r 

where r is such that 



£y 1 .y 2 ».«* » ^r 3 



2 P[Y = y I HI] = a' . 
i=l 1 u 

(He again assume a* is chosen so that it is a realizable 



value.) The power n' is then 2 P[Y="y I Hj], 

i=l 1 1 
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There are several schemes for reducing the work re- 



quired to produce the critical region (or equivalently, the 

acceptance region). The most straightforward is to find 

and compute p If P [?= y^ | H* ] < a ' , find "y^ 

and compute P [Y=*yJ H']. If 2 P [y = y. I H' ] < a * 

s 1 _l i=l 1 0 



y, etc., until 2 p [Y="y I H'] = a*, 
* i=l 1 0 



, find 

at which point s = r 
and the critical region is defined. An identical procedure 
may be used to define the critical region for a test of 
specified power by using the distribution of Y under H’ 
rather than Hq. An even faster method (in most practical 
cases) is to start by finding y^ and working "down" to the 
required size (or power) in a similar manner. 

A computer code (Appendix B) has been written which 
can complete a problem with n=20, k=5 (5 different first 
stage tests) in less than two minutes on a medium sized third 
generation computer. This program will give the acceptance 
region and power of a test of some given size or if desired, 
will print out all or a portion of the cumulative distri- 
bution function of Y under both Hq and H-p 

In this test, as with the identical first stage tests, 
there is some optimal choice of the values for a , j=l ,2, k 
which will maximize tt' for a given size a*. The appropriate 
program is 



r . 

Maximize tt' = 2 P [Y = \ 17 ( a ) J 

i=l 
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Subject to: 



2 PLY=yJa] = a* and \(y* ) < \(y* ) 

i=l ■ Ll 1 l+l 

for all i, i=l,2,...,v-l. 

This is a challenging program indeed and we shall not 
discuss the problem here, except to state that in the proto- 
type situation it was possible to obtain powers approaching 
those available under the assumption of equal performance, 
using an ad hoc procedure (Chapter 4). 

As a final comment we note that one sided tests and 
tests with composite hypotheses may be made if appropriate 
first stage tests are available. The derivation of the 
specific two stage test for such cases can be completed in a 
manner analogous to the foregoing. 
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CHAPTER IV 



ADAPTING THE TWO STAGE TEST 
TO THE PROTOTYPE SITUATION 

We now consider the application of the two stage test 
to our prototype testing situation. We wish to test 

HqJ d^ = f Vi, against (say) 

Hj^i d^ — f — V i » i = 1 ,2 , . . . ,n. 

The use of the two stage procedure is facilitated by 

specification of the A^s in such a manner that Fisher's 

Exact Test can be used as a first stage test. Thus we will 

specify so that is a constant for all d^ and f 

which satisfy d, - f . = A. 

i 1 i 

Let us consider equation 19 of Chapter II, which 

may be used to compute the power of Fisher's Exact Test. 

We consider now, however, the application of Fisher's test 
th th 

to the i individual. For the i individual, equation 19 
becomes 

p E J i - Jil L i = w- 

1 

~b d 1 (l»f 1 JV-Ji. ! ( l i - j ± ) ! (m-j i ) ! (m+j i ~l i ) ! ^ 

r=a f i (1-^ ) *r\ (i i -r) i (m-v ) ! (m+r-i ) i 

where a = max (»• and b = min & hi- 



39 




0.0 d l.o 

i 

FIGURE 6 

Some Sample Specifications of 
Let us specify 

h i . : a - f 1 rr ■ 

11 1 1 [f -(D-l)] +1 

wnere D € (0, °«). 

A graphical interpretation of this specification is 
shown in Figure 6 for several values of D. 

Two interesting features emerge from this specification. 
First, as seen in Figure 6, this specification for tends 
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to satisfy one's intuitive feelings about a reasonable form 



for H 1 . . That is, we feel that the difference d. - f . , about 
H 1 i 

which we are concerned, can be relatively large for f at 
midrange, but should shrink as f approaches its extreme 
values. Second, solving d. - f = (1-f . )/ [f (D-l )] +1 for 

4,(l-f.) d.U-f,) 11 1 1 

-rr=-n — r — r yields - — 7 - — - — r = D, and equation 1 may be written 
f. (l“ d . ) ( 1 -d. ) 

11 11 



P [ J i = J i| L i = 1 i’ H l> 



b r - 

2 [D] 

*=a L 



Oi ! ) * (m-l-Ji- 



-y ! ( 1 ^- nr ) j (m- ) ! (m+V 



--V J 



The power of the (first stage) test is then 



( 2 ) 



rr = 



b 

2 



1 j i =j(+a+l £ V-Jj. I (lj.-Jj.)! (m- J ± ) ! (m-i- J i -l i ) ! 

-v=a * -r ! (l i - -r) ! (m- nr ) ! (m+r-^ ) ! 



(3) 



where is such that 



P [J 1 >3 1 |X. 1 = l 1 ,H 01 ] = 



b 
2 

J .=j’+a+l 
1 i 



m m 

(J A 



(?“> 

i 



= a 



(4) 



Should we desire a two sided test we can specify 

1 -f, 



H 



li * 



d - f 
i i 



[f. (D-l )]" 1 +1 



d, ( 1 -f, ) 

Solving for f 1 ■ .^-1-- in this specification of H.^ yields 

the solutions^ ani 1/D. It may be readily verified that 



equation 3 yields the same value of tk when 1/D is substituted 



4l 



for D. Hence, the power of this (two sided) test is still 
given by equation 3 provided the value of j ’ is set by 



■ p C J i > J il L i ■ V H oi> p C J i < Vhl h - \ > H oi ] ' 



b 

2 



<? ) 

J i x i J i 



i =j’+a+l (^ m ) 

1 J i 



( 5 ) 



THE TWO STAGE TEST FOR THE PROTOTYPE CASE 

life now discuss our test of the hypotheses 

H : d = f Vi against 
0 i i 



H 




— i_ Vi, D €(0,~). 

[f.(D-2)] _1 .H 



The first stage tests T 1 ,T , ...,T will be, for each 

X ^ n 

1, i=l,2,...,n, Fisher’s Exact Test of the hypotheses 



H : 
Oi 



d 

i 



1 - f < 



f against H u : & x - f , = ^ 7TTT 



1 [^(D-l)]” 1 *! 



Given the observed values 1 = (1 ,1_,...,1 ) of 

1 2 n 

L = (L, ,L 0 * . . . ,L ) (total indicated detections by each 
12 n 

subject) we form (in general) m groups of results by the 
rule "group V contains all results where 1^ =*y~ or 1^ = 2m-V, 
V= 1 ,2 , . . . ,n. " Because Fisher’s test is the same for 1^ =y 
or li = 2m -*v* , the tests to be applied within each group 



42 



are indeed identical. Continuing in the notation of Chapter 
3, the number of results (or equivalently, the number of 
first stage tests) in group V will be denoted 0-y. for 
v=l,2,...,m. Of course some of the 0£s may be zero, and 
while the derivation of the two stage test did not take 
into account that possibility, it is easily shown that any 
number of the 0^.* s may be zero without affecting the pro- 
cedure or its power. It is convenient here to set k, the 
number of different tests, equal to m and allow some of the 
0^' s to be zero, as necessary. 

We may use the computer code of Appendix A to compute 
the available values of a v and tt^, that is, the possible 
non-randomized sizes and corresponding powers of the (first 
stage) tests in each group. We then choose, by a method 
discussed below, a size for the tests in each group and 
apply Fisher's test at that size to all results in that 
group, for each group ”>* = l,2,..„,m. The computer code 
of Appendix B may then be used to obtain a characterization 
of an acceptance region for the second stage test at the 
desired size a’. Finally we form the test statistic 
Y = ( Y^ , Y^ , . . . , Y n ) where Y->- is the number of rejections of 
Hq.^ from among the 0-y first stage tests applied to the 
results in group . If Y is in the acceptance region, we 
accept H q , otherwise we reject H Q and the procedure is com- 
plete. 

It is interesting that the power of the two stage test 
just described is a random variable. This is true because 
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in the equation 



rr 




( 6 ) 



the values of ct-r and n- r (a r .) depend upon the realization of 
L, (or more specifically, the values of 0^, 1 ,2, . . . ,m). 

We note further that the concept of expected power cannot 
be applied here, since our assumptions preclude specifica- 
tion of a distribution for L. (Indeed, suppose we did 
assume a distribution for L. Then we could (in theory) 

derive the distributions of J. and L. - J. , which would be 

i i l 

logically equivalent to fixing values of d^ and f^, violat- 
ing our assumptions. ) 

However, even when the power of the two stage test 
obtains its "poorer" values, it still has good power for 
practical purposes, and in fact that power approaches the 
power available from tests made under the assumption of 
equal performance by all individuals. We next consider the 
choice of the best set of sizes for the first stage tests. 

CHOOSING FIRST STAGE SIZES 

As noted in Chapter 3» the "optimal" choice of first 
stage sizes hv, Y*= 1 ,2 , . . . ,m would be given by a solution 
to the program 



r 



2 pCr="y.| n(a)] 
i 



Maximize tt* = 



i=l 
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Subject to: £ p[Y = y aj = <x • and 

i=l i 

My^) <A(y 1+1 ), i = 1 , 2 ,... ,v - 1 

where v = 77 (0.+1 ) , a = (ai ,a„ , . . . ,a ) and 
1 = 1 * m 

*(“) = ( n i ( a J» n 2 ( a o )»•••»' n ( a ))• 

We have been unable to solve this program in general, 
so we offer only a rule of thumb for choosing the sizes 
of the first stage tests. This practical approach has been 
successful in the cases examined thus far. The rule in- 
volves use of non-randomized values for only , and only 
assures a "near optimal" vector a w = ( a 2 W ’ a 2w’ * * * ,a mw^ 
where by "near optimal" we mean that only minor improvement 
in the value of rr • (say in the second or third decimal 

place) would accrue if the true optimal vector a were 

o 

found. 

The first step involves finding an upper bound for 

rr'. Consider the result 0„ = 0 ,v= 1 ,2 , . . . ,m - 1 , 0 =n, 

m 

which represents the most "informative" result obtainable. 
Since such a result reduces to the identical first stage 
tests situation, we would be testing the second parameter 
of a binomial distribution, that is 

H n : p = a m against Hi : p = rr . 

u m m 
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The best choice of (a ,rr ) can be obtained for that case 

m m 

by enumerating the powers i T' obtained from each of the 
available (o^.,tt^.) pairs, using tables of the binomial 
distribution. Let 77^* be the highest such power obtained 
for the desired size a'. 

Now, using the computer code of Appendix B, we may 
compute TT * for the vector a (a. ,a 0 ,...,a ) where 

3- 3 13 ^a uigL 

a , -r* = 1 ,2 , . . . ,m denotes the largest non-zero, non- 

va 

randomized value of a r less than or equal to 0„5» For a 

two stage test of size . 05 , if tt * is within (say) .02 of 

tt u * , it seems reasonable to accept a as "near optimal" 

since any other choice of a will improve tt* by .02 at most. 

If tt • is not within .02 of tt * , there is cause to believe 

a u 

it is not "near optimal" and a new vector "a^ should be 
tried. No firm suggestion for choosing is given, but 
the following observations, made from the experience of the 
case study to follow, are given as guidance. 

Components of a for which 0 r is large seem 
to be most sensitive (that is, a change of one "step" 
in available a r values causes a relatively large 
change in tt*). Components of a for which 0^. is small 
(say 1) seem quite insensitive. Thus in seeking the 
optimal a it would seem advisable to change the a^-'s 
for which 0 ^ is large first. 

2. It appears that each component may be "near- 
optimized" separately. That is, if 0 is the largest 
0~v value, then one may find the best value of for 
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arbitrary values of the other a r 's. Having found a 

"near optimal" value for a , say a . then one can 

q qo 

find a "near optimal" value for a p , say a p 0 » the 

vector (a. ,a 0 , . . . ,a . .... a , ...) where a-r is arbitrary 
1 p qo 

for all except p and q and 0^ is the next largest 

This procedure may be continued until all compo- 
nents of a have been "near-optimized". Due to inter- 
actions between components of a the resulting vector 
may not be optimal but the case study experience 
provides strong evidence that it will be near optimal. 

tfhile the foregoing seems to suggest a lengthy procedure, 
in a very limited set of test cases considered, the initial 
vector a a was "near optimal" and no further work was 
necessary. Thus it may be that the choice of a is not a 
problem in practice. 

A CASE STUDY 

Our prototype experiment was based on an experiment 
conducted at the US Naval Postgraduate School in the spring 
of 1968. There were n=21 subjects, each of whom made 8 
attempts to detect actual targets and 8 attempts to detect 
non-existent targets (m=8). The value D= 3 was used in the 
specification of H^, making the hypotheses under test 

v v f i yi agalnst Hl : d i - f i = [ 2 _ f‘ ]= i;r Vl - 

The experimental data is given in Table 2. 
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TABLE 2 



Results of a Prototype Experiment 



Subject 

Number 

(i) 


Indicated 

Detections 

<V 


True 

Detections 


Subject 

Number 

(i) 


Indicated 

Detections 

( V 


True 

Detections 

‘V 


1 


12 


5 


11 


2 


1 


2 


7 


5 


12 


3 


2 


3 


10 


5 


13 


10 


5 


4 


10 


4 


14 


n 


4 


5 


7 


5 


15 


7 


3 


6 


9 


5 


16 


14 


8 


7 


15 


7 


17 


12 


6 


8 


12 


5 


18 


10 


6 


9 


14 


8 


19 


7 


3 


10 


7 


3 


20 


7 


3 








21 


9 


4 
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A near optimal vector a = (a . ,a_ , » » e ,a Q ) was arrived. 

1 c o 

at using the procedure discussed previously. It may be 

informative to reconstruct that procedure at this point. 

The (hypothetical) case 0 V =O, V = l ,2 , . „ . ,7 ,0 O = 2l was used 

o 

to get an upper bound tt u * on the power for size a* = . 05 . 

The various (a r »nr) pairs available for each group •v, 
T’=i 1 2j...j 8 were obtained using the computer code of 
Appendix A, and are reproduced in Table 3» along with a 
listing of the total responses (1^ * s ) which fell into each 
given group V according to the rule given earlier (group V 
contains all results or = 2m -V) and the critical 

value for conducting the (first stage) test at the listed 
size a when the total response is 1^. tt u * was found to 
be „994 and the first trial vector a was (.50, .23, .50, .28, 

SL 

.50, .30, .50, .31 ). Use of a yielded tt ' = .98401 which is 

Q, Ql 

near optimal. The first stage tests were then applied at 

the sizes in a with results as shown in Table 4. The 
a 

vector Y = (Y , Yg» « . . , Yg ) = (0,2 ,1 ,0 , 0 ,1 ,3,0 ) , which may be 
read directly as the right hand column of Table 4, was 
found to be in the acceptance region and Hq was accepted 
at the .05 level of significance. Note that the computer 
output from the trial computation for a includes a complete 
description of the acceptance region, so further compu- 
tation was not necessary. 

ADDITIONAL RESULT S 

Although not mentioned in the section on choosing a 
near optimal first stage size vector a, there is evidence 
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1 



TABLE 3 



Available (a.*, n^.) Pairs for the Prototype Experiment 



Group Number 


(a v , it^) 


l. 


Critical Value C 


'(-r) 


Reject for > C 


1 


(.50, .75) 


l 


0 






15 


.. 7 


2 


(.23, .53) 


2 


1 






14 


7 




(.50, .83) 


3 


1 


3 




13 


6 




(.10, .36) 


3 


2 






13 


1 




(.28, .68) 


4 


2 


4 




12 


6 




(.04. .22) 


4 


3 






12 


7 




(.50, .86) 


5 


2 






11 


5 


5 


(.14, .50) 


5 


3 






11 


6 




(.01, .12) 


5 


4 






11 


7 




(.30, .74) 


"IT 


3 






10 


5 


6 


(.06, .33) 


6 


4 






10 


6 




(.003, .05) 


6 


5 






10 


7 




(.50, .88) 


7 


3 






9 


4 




(.16, .56) 


7 


4 


7 




9 


5 




(.02, .18) 


7 


5 




9 


6 




(.001,. 02 ) 


7 


6 






9 


7 




(.31, .75) 


8 


4 




(.07, .36) 


8 


5 


8 


(.005, .07) 


8 


6 




(.0001, .003) 


8 


7 
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TABLE 4 

Results of First Stage Tests 



Group Number 
( T* ) 



Number of First 
Stage Tests in 
Group ( 0 ^.) 



Size Used to 
Perform Tests 



(<Xr) 



Number of 
Rejections 

(Y-r) 



1 1 

2 3 

3 1 

4 3 

5 i 

6 4 

7 8 

8 0 



o50 o 

.23 2 

.50 1 

.28 0 

.50 0 

o30 1 

*50 3 

.31 0 



that a rather large number of near optimal vectors (a) 
exist. In the case study just presented, an extended in- 
vestigation revealed that there are at least 24 near optimal 
vectors for a test of size .05. (Indeed there may be more 
than 24, since the investigation did not enumerate all 
possible vectors a. ) Table 5 presents a resume of the near 
optimal vectors found. It is interesting that at least 20 
first stage size vectors yield powers which differ by less 
than e 004 o Additionally, the investigation disclosed at 
least 26 more vectors which produce powers greater than .974 
and may be considered "near-optimal". 
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TABLE 5 

A List of Near Optimal First Stage Size Vectors 



Notoj ex ^ = * ^2 — ° 


23, and a 


q = .31 in all vectors. 


Only 


the subvector 


( a 3>V a 5 


,a.,a ) is tabulated. 
0 7 




Subvector 


TT 1 


Subvector 


TT ' 


(.50, .28, .14, .30, .16) 


.98435 


(.10, .28, .01, .30, .16) 


.98138 


(.50, .28, .50, .30, .16) 


.98419 


(.10, .28, .01, .30, .50) 


.98054 


(.50, .28, .50, .30, .50) 


.98401 


(.50, .72, .14, .30, .16) 


.98039 


( .50, .28, . 14, . 30, .50 ) 


.98391 


(.50, .72, .50, .30, .16) 


.98035 


(.10, .28, .50, .30, .16) 


.98382 


(.90, .28, .86, .30, .16) 


.98015 


(.10, .28, .114-, .30, .16) 


.98360 


(.50, .72, .50, .30, .50) 


.9799 4 


(.10, .28, .50, .30, .50) 


.98344 


(.50. ,72, .14, .30, .50) 


.97971 


(.10, .28, .14, .30, .50) 


.98288 


(.90, .72, .50, .30, .16) 


.97829 


(.90, .28, .50, .30, .16) 


.98234 


(.50, .04, .14, .30, .16) 


.97821 


(.50, .28, .86, .30, .16) 


.98232 


(.50, .04, .50, .30, .16) 


.97782 


(.90, .28, .14, .30, .16) 


.98220 


(.90, .72, .14, .30, .16) 


.97768 


(.90, .28, .50, .30, .50) 


.98205 


(.50, .28, .50, .70, .16) 


.97750 


(.50, .28, .86, .30, .50) 


.98190 


(.10, .04, .50, .30, .16) 


.97745 


(.90, .28, .14, .30, .50) 


.98174 


(.50, .72, .86, .30, .50) 


.97743 


(.50, .28, .01, .30, .16) 


.98169 


(.50, .28, .50, .06, .16) 


.97741 


(.50, .28, .01, .30, .50) 


.98161 


(.50, .28, .14, .06, .16) 


.97738 
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If we consider an experidment somewhat smaller than 
in the preceeding case study, it is possible to list all 
possible first stage size vectors (easily) and compute the 
resulting powers tt’. Let us consider the case n=l3, m=8, 
and suppose the total responses (l^'s) were such that 
= 0 ,y" = 1,2, 3*4, 8, 0^ = 1, 0^ = 4, and 0^ = 8. In this 
instance the computer code of Appendix 2 produced values of 
rr* for all possible first stage size vectors in less than 
1 minute on the IBM 360 computer. The optimal vector cc^ 
yielded a power of .949® For small experiments, or when 
less than 5 of the 0's are non zero, this complete enumer- 
ation technique is recommended. For other cases, the number 
of possible vectors (a’s) is generally rather large and 
computation time grows rapidly. In the case study presented, 
a complete enumeration would have required more than 4 hours 
of computer time, which is prohibitive for most users, 
especially in view of the small improvement in tt* which may 
result. 

COMMENTS ON POWER 

In formulating the two stage test, we have weakened 
some of the assumptions required for the earlier techniques 
considered in Chapter 1. We would therefore expect some 
reduction in power compared to the tests which take advantage 
the stronger assumptions. In order to give some clarifi- 
cation of this point, a numerical example is now given. 
Suppose the experiment discussed in the case study had been 
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analyzed under the additional assumption that all individuals 

perform equally. Under that assumption, the total number of 

21 

true detections E j =97 and the total number of false 

21 1=1 i 

alarms E 1 * - j. = 98 would have been considered as obser- 
i=l 1 1 d 

vat ions on the random variables J' and K* where J* = b(l 68 ,d) 

and K' = b(l 68 ,f). Typically, the test of H Q : d=f against 

1 - f 

H 1 : d-f=- 3 would be made. Using one of the usual 

1 [ 2 f] _1 +l 

asymtotic techniques2 with size .05 results in a power of 

.999 at the point (d= ,75> f = * 5 ) a typical point on the 

curve used in the alternate hypothesis for the case study. 

Comparing .999 with . 984 , the optimal power for the two 

stage test in the case study, we see that relaxation of the 

assumptions d^ = d Vi, and f ^ = f Vi has "cost" .015 in 

power. While it is for the individual experimenter to 

decide on the merits of this "trade off", it seems that in 

general, the loss of power is slight in light of the much 

broader applicability of the two stage test. Additionally, 

at the point (d=.9> f = .75) which is also on the curve 

d-f = — — » the asymtotic technique yields power of .980 

[2f ]~ -hi 

whereas the two stage test continues to have power of . 984 . 

At this more extreme point, the two stage test is better 
than the asymtotic test under the stronger assumptions. 

It may also be important to set a lower bound on the 
power of a two stage test for a given experiment before it 
is run. The poorest power would emerge if all results fell 
in group 1 , that is, if was 1 or m - 1 for all i. How- 

ever, the result 0^ = n is so unlikely that it is felt the 
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hypothetical result 0 = n or even 0 = n can be used to give 
a practical lower bound for power which is still rather 
conservative (perhaps even pessimistic). 

In the case study, if 0 ? was 21, the test H : p = ct 
against : p = rr^ would be used to determine the lower 
bound on -rr* » where (c^tt^) is the size - power pair which 
gives the best power at size .05. Completing this procedure 
for the case example yields .91 as practical lower bound on 
the power of the two stage test. Of course, the actual 
results were much more "informative" than 0^ = n, and the 
actual power achieved was .984. 
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CHAPTER V 



A RAPID APPROXIMATE PROCEDURE 

If we strengthen assumption 1 (p. 11) by adding the 
statement "m is sufficiently large to assure that the arcsine 

g o 

transformation *' provides reasonably 'normal' random 
variates when applied to the experimental data", (leaving 
all other assumptions as stated), we may then use an approx- 
imate procedure to test the hypothesis 

H q : d 1 = f i Vi against : d^ ^ f Vi. 

Only a brief derivation will be given and we make no 
comments about the properties of the procedure except to 
remark that it is easy and rapid in application as compared 
to the two stage test, and in all test cases thus far 
examined, has provided the same results (in terms of re- 
jection of Hq) as the two stage test. 

DERIVATION OF THE PROCEDURE 

z/e shall use the terminology of Chapter 2. Under the 
strengthened version of assumption 1 noted above, we have 
(approximately ) 

- - - ^ 

arcsin Yj^/m = N(arcsin yd^ , 1/km) Vi» and 
arc sin y K /m = N (arcsin -/f^ , 1/km) Vi. 
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Prom the ivell known properties of the normal distribution 



2m arcsin J^/m = N( 2m arcsin^d^ , 1/2) V i, and 
2m arcsin J K^/m = N( 2m arcsin-JT^, 1/2) yi. 

Further, for each i, i=l,2,...,n 



2m 



(arcsin arcsin ^ K^/m ) = N( 2m Larcsln^d^ - arcsin , {f^],l). 



Now under H : f . = d V i, we have 
0 i i 



2m 



(arcsin - arcs in Y I^/m) = N(0,1) Vi, 



Denoting 2m (arcsin -yj^/m, - arcsin K^/m ) by Z^ , 



we have 



( Z . ) — > i — l,2,o. c.,n, 



so that 



e ( z .) 2 a ** . 

1=1 1 n 



In order to test H : d = f . Vi against H. : d. / f Vi at 

u i r i i 

n 2 0/2 

size a, we may reject H if £ (Z ) > {“T \ „ 

0 1=1 i n > i-a 
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APPENDIX A 



A FORTRAN CODE FOR COMPUTING OPERATING CHARACTERISTIC 
CURVES FOR FISHER'S EXACT TEST OF PERCENTAGES 



The code which follows evaluates equation 19 of 
Chapter 2 appropriately to form the complementary cumulative 



distribution functions of J\ 



1 for each 1^, 1^ = 1 ,2 , . . . ,2m 



under both the null and alternate hypothesis. The input 
parameters PD, PF, and M correspond respectively, to d^, 
f^, and m in equation 19. The values of PD and PF used as 
input to the code can be any point on the alternate- hypothesis 
curve d^ - f^ = (1 - f ^ )/ £ £f^ (D - 1 )] ^ & 1 j 1 . (The program 
is useful, of course, for any investigation of Fisher's 
Test whether or not related to the specific use intended 
here. ) 



A subroutine format is used to allow maximum flexi- 
bility in the application of the code. Depending on the 
computer used, this subroutine should perform for values of 
m up to at least 40. The output includes a full definition 
of parameters and variables and is self-explanatory. 
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SUBROUTINE HYPER ( M,PF , PD) 

P=PD*(i-PF)/PF/!l-PD) 

0=1. O/P 
WRI TEI6.4) 

4 FORMAT! 1H1,4X, 'DEFINITION OF SYMBOLS: ' »//»10X, ' M IS' 
1* THE NUMBER OF DETECTION ATTEMPTS MADE UNDER EACH • 
2 ' TEST CONDITION. •// ,1GX, *L IS THE TOTAL NUMBER OF • 

3 • I NDI CATED DE TECT I ONS. ' // »10X , • J IS THE NUMBER OF ' 

4 ' CORREC T DETECTIONS. '//,10X, 'SO IS THE PROBABILITY • 
5 'OF OUTCOMES STRICTLY GREATER THAN J UNDER HO.'//, 
610X, • SI IS THE PROBABILITY OF OUTCOMES STRICTLY • 

7 'GREATER THAN J UNDER HI. '//I 
WRITE(6,8)M,PF,PD,PF 

8 FORMAT! 5X. 'DISTR IBUTION FOR M=* I 2 , • . • / / , 5 X , • HO : • 

1 ' PROBABI L I TY OF FALSE ALARM = PROBABILITY OF TRUE • 

2 'DETECT I ON = 'F6.4 ,'.'// ,5X, 'HI : PROBABILITY OF TRUE' 

3' DETECTION ='F6.4,'» '//,9X,' PROBABILITY OF FALSE • 

4 ' ALARM = • F6. 4 , ' • '//) 

WRI TE ( 6 , 5 ) 

5 FORMAT! 5X, • L • 8 X , • J ' 9X , • SC ' 1 3X , • S 1 • 1 2X , • L ' 8X , • J • / ) 

DO 50 L=1 , M 

WRI TE ( 6 ,7 ) 

7 FORMAT! IX, //» 

SC=1.0 
Sl=1.0 
L L = L+ 1 

DO 50 J J=1 , L L 
J=JJ-1 
X JF=1. 0 

IF! J-l ) 11,11,9 

9 DO 10 K=1,J 
XK = K 

XJF=XJF*XK 
CONTINUE 
XL JF=1 . 0 
L J=L- J 

IF(LJ)14,14,12 
DO 13 K=1,LJ 
XK = K 



10 

11 



12 



XL JF=XL JF*XK 

13 CONTINUE 

14 XM JF=1. 0 
M J=M- J 

IF! MJ) 17,17,15 

15 DO 16 K=1 , MJ 
XK=K 

XMJF=XMJF*XK 

16 CONTINUE 

17 XMJLF=1.0 
M JL=M+ J-L 
IFIMJL-1) 20,20, 18 

18 DO 19 K=1 , M JL 
XK = K 

XMJLF=XMJLF*XK 

19 CONTINUE 

20 W=0.0 
U=0.0 

DO 40 KKA = 1 ,LL 

KA=KKA-1 

XAF=1.0 

I F ( KA-1 123,23,21 

21 DO 22 K=1,KA 
XK=K 

XAF=XAF*XK 

22 CONTINUE 

23 MA=M— KA 
XMAF=1.0 

IF(MA-1)26,26,24 

24 DO 25 K=1 , MA 
XK=K 
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XMAF=XMAF*XK 

25 CONTINUE 

26 LA=L-KA 
XLAF=1.0 

IF(LA-1)29,29,27 

27 00 28 K-l , LA 
XK = K 

XLAF=XLAF*XK 

28 CONTINUE 

29 MAL=M+KA-L 
XHA LF = l* 0 
P0=1.0 

IF ( HAL- 1132,32 ,30 

30 00 31 K=1,MAL 
XK =K 

XMALF=XMALF*XK 

31 CONTINUE 

32 IF(KA-J)33,35,34 

33 PQ=Q**( J-KA ) 

GO TO 35 

34 PO=P**(KA-J» 

35 UA=( XJF/XAF)*(XLJF/XLAF)*(XMJF/XMAF»*(XMJLF/XMALFI 
PQ=PQ*UA 

U=U+UA 
W=W+PQ 
40 CONTINUE 
R=l.0/W 
S=1.0/U 
ML=2*M-L 
SO=SO-S 

I F ( SO I 60 » 61 ,61 

60 S0=0.0 

61 Si=Sl-R 

IF( SI 162,63,63 

62 S1=0.0 

63 WRI TE(6,6) L, J, SO, Sl.ML.MJl 

6 FORMAT! 4X, 12, 7X,I 2,7X,2 (F10.8,5X) , 12, 7X, 12,// ) 

50 CONTINUE 
END 



APPENDIX B 



A FORTRAN CODE FOR DETERMINING THE ACCEPTANCE REGION 
FOR THE SECOND STAGE TEST IN THE TWO STAGE PROCEDURE 



yh 



In general, this program can be used, to generate the 
CDF's for two joint distributions of up to 12 independent 
binomial distributions with first parameters given by 
(KPHI(l), KPHI(2 ) , . . „ ,KPHI(12 ) ) in both cases and second 
parameters given by ( ALPHA ( 1 ) , ALPHA ( 2 ) , . „ , , ALPHA (12)) for 
the first joint distribution and by (XPI(l), XPI(2 
XPI(12)) for the second,, 

The program input parameters which relate to the 
prototype case variables are: 

KPHI(I) - 0, Number of first stage tests in the i* 1 
group o 

ALPHA(I)~a^ Size of first stage tests in the i' 
group . 

XPI(I) - tt power of first stage tests in the i' 

1 

group e 

M - k Number of groups which contain at least 

one test. (If desired, all groups, in- 
cluding those for which 0^ = 0 can be read 
in to the program but this practice 
increases computing time. If this prac- 
tice is followed, then M corresponds 
to m 9 ) 



th 



th 
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Other program input parameters are: 

START The largest size for which a printed value of 
a' and n' is desired. 

SIZE The smallest size for which a printed value of 
a' and n* i.s desired. 

KP The Kind of print parameter. 

KP = 0 provides a complete printout of sizes and 
corresponding powers for all sizes between SIZE 
and START plus a summarized description of the 
acceptance region for a test of size SIZE. 

KP=-1 provides only the print out of sizes and 
corresponding powers for ail sizes between SIZE 
and START. 

KP = 1 provides only the summarized description 
of the acceptance region for a test of size SIZE 
plus a statement of the power of that test. 

NOTE: Regardless of the KP parameter specified, 

the output also always includes a recap of the 
0 ^’s, a^'s, and rr^’s for verification purposes. 

In addition to reading in the above parameters, it is 

also necessary to insure that the dimension of KSUM, XLAM, 

and KX is greater than or equal to v, which can be computed 

m 

by the formula v= 7 j (0. +1). With KSUM, XLAM, and KX so 

i=l 1 

dimensioned, the program will operate on any experiment for 
which the number of subjects (n) is less than or equal to 
30 and for which k, the number of non-zero 0 ^'s is less 
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than or equal to 12 „ The program is readily extended to 
larger problems but experience indicates that such cases 
cause excessive computing time and/or exceed computer 
storage capacity. 
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non ooo non oono non 



I NTEGER*2 KSUM.KX 

DIMENSION KPHI (l2).A!21,21) » KT ( 31 ) » ALPHA! 12) , 

1ABAR! 12) .XPI ( 12), PI BAR! 12) ,XL( 12) , XL BAR! 12),LL< 12) , 
2 ML ! 12) » PA (21 ,21,12) ,NY( 31 ) ,KFY( 31) , 

3XLAM! 5760 ) ,KX(5760) ,KSUK(5760) 

READ! 5,2 >M, START. KP.S I ZE 
2 F0RMATU8.F8.0,I8,F8.0) 

READ (5.1 ) (KPHI (I) .ALPHA! I ) , XPI ( I ) , 1=1, Ml 
1 FORMAT! 18, 2F8.0) 

INITIALIZE ARRAYS AS NECESSARY 



DO 10 1=1,31 
NY! I ) =0 
KFY! I ) =0 

10 CONTINUE 

DO 11 1=1,12 
LL( I ) =0 

11 CONTINUE 

DO 16 1=1,31 
KT! I ) =0 
16 CONTINUE 
KYT=1 



COMPUTE AUXILIARY PARAMETERS AND ONE PLUS THE 
LARGEST POSSIBLE NUMBER OF SUCCESSES ( KYT ) 

DO 15 1=1, M 
A BAR! I ) =1.0- ALPHA! I ) 

PIBAR! I ) =1.0-XPI ( I ) 

XL! I )=ALPHA( I) /XPI (I ) 

XLBAR ( I ) =ABAR ( I ) /PI BAR! I ) 

KYT=KYT+KPH I ( I > 

15 CONTINUE 

SET UNNEEDED PARAMETRS TO THEIR IDLE VALUES 

IF! M- 12)17, 19, 19 

17 MM=M*1 

DO 18 I =MM , 1 2 
KPHI ( I ) =0 

18 CONTINUE 

COMPUTE THE NUMBER OF DIFFERENT VECTORS POSSIBLE 
N=1 

19 DO 3 1=1,12 

ML! I ) =KPHI ( I ) +1 
N=N*ML( I ) 

3 CONTINUE 

SET PARAMETERS NEEDED FOR ENSUING DO LOOPS 

M1=ML ( 1 ) 

M2=ML ( 2 ) 

M3=ML(3) 

M4=ML(4) 

M5=ML( 5) 

M6=ML(6) 

M7=ML( 7 ) 

M8=ML(8) 

M9=ML ( 9 ) 

M10=ML ( 10) 

M11=ML( 11) 

M12=ML( 12) 

WRITE! 6.200 ) 

200 FORMAT! 1H1 ,//) 

NPLUS=N+1 
JT=ML ( 1 ) 

DO 700 I =2 , M 
IF! JT-ML! I )) 701 ,700,700 
701 JT=ML ( I ) 
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700 CONTINUE 



COMPUTE COEFFICIENTS FOR BINOMIAL PROBABILITY 
EXPRESSIONS 

DO 20 1=1, JT 
X I = I 

or 20 j=i , i 

XJ = J 

XJBAR=XI-XJ*1.( 

A ( I , J ) =GA MMA ( X I ) / (GAMMA <XJ ) *G AM MA( XJBAR ) ) 

MULTIPLY COEFFICIENTS BY APPROPRIATE POWERS OF "P" 
AND "Q". PA ( I , J , K ) IS THE PROBABILITY OF J SUCCESSES 
GIVEN I TRIALS WITH SUCCESS PROBABILITY ("P") EOUAL 
TO XPI (K|. 

IF ( J-l) 5,5,6 

5 I F ( I - J I 7 , 7 , 8 

7 DO 21 K=1,m 

P A ( I , J , K ) =A ( I , J) 

21 CONTINUE 
GO TO 20 

8 DO 22 KK=l f M 

I F ( I -ML ( KK ) > 301 ,301 ,22 
301 PA< I , J,KK)=A( I , J)*(PI RAR(KK)**( I - J ) I 

22 CONTINUE 
GO TO 20 

6 I F ( I-J)9,9,4 

9 DC 23 KKK=1 , M 

I F ( I-ml(KKK) ) 302,302,23 
3G2 PA( I ,J,KKK)=A( I , J ) * ( X P I (KKK)**< J-l) ) 

23 CONTINUE 
GO TO 20 

A DO 24 KKKK= 1 , M 

I F ( I-ML(KKKK) >303,303,24 
303 P A ( I , J,KKKK)=A( I , J)*( XPI (KKKK)**( J-l ) )* 

1 ( PIBAR( KKKK ) ** ( I-J) ) 

24 CONTINUE 
20 CONTINUE 

COMPUTE LIKELIHOOD RATIOS FOR ALL VECTORS (XLAM(L)I 
L =0 

DO 90 J1 =1 , Ml 
K 1 = J 1- 1 
KB1=M1-J1 
I F ( K 1 ) 30 » 3C , 31 

30 B=XLBAR( 1 ) **K81 
GO TO 34 

31 IF(KB1 >32,32,33 

32 R = XL ( 1 > **K 1 
GO TO 34 

33 B=(XLBAR(1)**KB1)*(XL(1)**K1) 

34 DO 90 J2=l ,M2 
K2= J2-1 
KB2=M?-J2 
IF(K2)35,35,36 

35 I F( KB2 >91.91,37 

37 C=B*XLBAR(2>**KB2 
GO TO 40 

36 IF(KB2)38»38,39 

38 C=B*XL( 2) **K2 
GO TO 40 

39 C = B*( XLBAR(2>**KB2)*(XL(2)**K2) 

GO TO 40 

91 C=B 

40 DO 90 J3 = 1 ,M3 
K3 = J 3-1 
KB3=M3-J3 
IF(K3 >41 ,41 ,42 

41 I F( KB3 >92,92,43 



66 



43 D=C*XLBAR(3)**KB3 
GO TO 46 

42 I F( KB 3 ) 44,44 ,45 

44 D=C*XL(3)**K3 
GO TO 46 

45 D=C*< XLBAR(3»**KB3 )*<XL(3 )**K3> 

GO TO 46 

92 D=C 

46 DO 90 J4=1,M4 
K4=J4-1 
KB4=M4-J4 
IF(K4)47.47.48 

47 IF(KB4)93,93,49 

49 E=D*XLBAR(4) **KB4 
GO TO 52 

48 I F ( KB4) 50,50,51 

50 E=0*XL(4)**K4 
GO TO 52 

51 E=D*(XLBAR(4)**KB4)*(XL(4)**K4> 

GO TO 52 

93 E=D 

52 DO 90 J5=1,M5 
K5= J5-1 
KB5=M5-J5 
IF(K5) 53,53,54 

53 IF(KB5)94,94,55 

55 BB=E*XLBAR(5»**KB5 
GO TO 58 

54 I F ( KB5) 56 1 56 1 5 7 

56 BB=E*XL(5)**K5 
GO TO 58 

57 BB=E*(XLBAR(5)**KB5)*(XL(5)**K5 I 
GO TO 58 

94 BB=E 

58 DO 90 J6=1,M6 
K6= J6- 1 
KB6=M6-J6 

I F ( K6 ) 59,59,60 

59 IF ( KB6) 95 , 95 ,61 

61 CC=BB*XLBAR(6)**KB6 
GO TO 64 

60 I F( KB6) 62 ,62 ,63 

62 CC=BB»XL(6)**K6 
GO TO 64 

63 CC=BB*(XLBAR(6>**KB6>*(XL<6)**K6) 
GO TO 64 

95 COBB 

64 DO 90 J7=1,M7 
K7= J7-1 
KB7=M7- J7 

I F ( K7 ) 65 ,65 ,66 

65 IF(KB7)96, 96,67 

67 DD=CC*XLBAR(7)**KB7 
GO TO 70 

66 IF( KB 7) 68,68,69 

68 DD=CC*XL(7>**K7 
GO TO 70 

69 DD=CC*(XLBAR(7J**KB7)*(XL(7)**K7> 
GO TO 70 

96 DD=CC 

70 DO 90 J8=1,M8 
K8= J8- 1 
KB8=H8- J8 

IF ( K8 1 71 ,71 ,72 

71 IFI KB 8)97, 97,73 

73 EE=DD*XLBAP(8)**KB8 
GO TO 76 

72 I F ( KB8 ) 74.74,75 

74 EE=DD*XL C 8 ) **K8 
GO TO 76 

75 EE»DD*(XLBAR( 8 ) **KB8) *( XL ( 8 1 **K8 1 
GO TO 76 
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97 EE=DD 

76 DO 90 J9=l ,M9 
K9= J9-1 
KB9=M9-J9 

I F ( K9 >77*77,78 

77 I F ( KB9)98»98,79 

79 FF=EE*XLBAP (9) **KB9 
GO TO 82 

78 IF(KB9)80, 80,81 

80 FF=EE*XL(9)**K9 
GO TO 82 

81 FF=EE*(XL8AR(9) **KB9)*(XL(9 )**K9) 

GO TO 82 

98 FF=EE 

82 DO 90 J10=1,M10 
K10=J10-1 
KR10=M10-J1 J 
IF(K10)83,8?,84 

83 IF< KB10) 99,99.85 

85 GG=FF*XLBAR< 10 ) **KB10 
GO TO 88 

84 I F ( K910 ) 86,86,87 

86 GG=FF*XL( 10)**K1G 
GO TO 88 

87 GG=FF*( XLBAR ( 10 ) **KB1 0 ) * ( XL ( 10 ) **K 10 ) 

GO TO 88 

99 GG=FF 

88 DO 90 J11=1,M11 
K 1 1=J1 1—1 

KB 1 1 =M1 1 - J1 1 
I F ( K 1 1 ) 240 ,240,241 

240 IF(KB11 >100,100,242 

242 R=GG*XLBAR(U)**KB11 
GO TO 245 

241 IF(KB11)243»243, 244 

243 R=GG* XL ( 11 )**K11 
GO TO 245 

244 R=GG*( XLBAR ( 11 )**KB11 )*<XL( 11 )**K11 ) 

GO TO 245 

100 R=GG 

245 DO 90 J1 2= 1,M12 
K 1 2= J1 2-1 
KB12=M12-J12 
IF(K12)246,246.247 

246 IF(KB12>101 ,101,248 

248 RR=R*XLBAR(12)**KB12 
GO TO 251 

247 I F ( KB 1 2 ) 249 ,249,250 

249 RR=R*XL< 1 2 ) **K1 2 
GO TO 251 

250 RR=R*( XLBAR ( 12>**KB12)*(XL(12)**K12) 

GO TO 251 

101 RR=R 

ENCODE THE VECTOR (K1,K2, .*.,K12> INTO THE NUMBER L 

251 L =L+ 1 

COMPUTE THE SUM OF COMPONENENTS IN THE VECTOR 
<K1,K2,... , K12 ) 

KY=J12 + Kll + K10*K9+K8 + K7 + K6 + K5+K4+K3 + K 2+K 1 
COUNT THE NUMBER OF VECTORS WHOSE SUM IS KY 
NY( KY> =NY( KY ) ♦ 1 

ENTER THE LIKELIHOOD RATIO AND THE ENCOOEO V4LUE 
OF ITS ARGUMENT INTO THEIR RESPECTIVE ARRAYS 

XLAM( L) =RR 
KX( L)=L 
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90 CONTINUE 

SORT THE LIKELIHOOD RATIOS INTO AN 0R0ERED ARRAY 

NMIN=N-1 
DO 380 l =1 » NMI N 
1 1 = 1 + 1 

DO 381 J=I I , N 

IF(XLAM(I)-XLAM(JI 1382.381,381 
382 T=XLAM(J> 

I T=KX( J I 

XLAM ( J I =XLAM( I ) 

KX( J I =KX ( I I 
XLAH( I > = T 
KX( I ) = I T 
381 CONTINUE 
380 CONTINUE 

PRINT GENERAL HEADINGS 

F = 1 • 0 

0 = 1.0 

WRITE(6.209MKPHI ( I), 1 = 1. MI 

209 FORMAT! 1H1 , ‘NUMBER OF TRIALS AND ASSOCIATED SUCCESS' 
1' PROBABILITIES UNDER THE NULL AND ALTERNATE HYPOTH' 
2'ESES , ///»3X» ' NUMBER ' / » IX , ' OF TRIALS: '3X, 12( 12,4X1 I 
IF! KP) 410.410,411 

411 WRI TE ( 6.4161 

416 FORMAT! 1X,//I 
GO TO 417 

410 WRI TE ( 6 ,405 1 

405 FORMAT ( IH+ ,90X , 'NOTE: THE BELOW L I S TED' / , 89X , • CUMUL ' 
I'ATIVE PROBABILITIES ARE' / ,89X, ' DEF INED AS THE PROS' 
2' ABILITY' I 

417 WRITE(6,208I (ALPHA(I I » I =1 » MI 

208 FORMAT! 5X, • ALPHA : • 2X , 12 ! F4. 2 , 2X I ) 

IF(KP)412»412»413 

413 WRI TE ( 6,416 I 
GO TO 418 

412 WRITE! 6.406 1 

406 FORMAT! IH+.88X, 'THAT THE VECTOR TO THE LEFT* /,89X, 

1 ' OR ANY BELOW IT (THAT IS,'/, 89X, • W ITH SMALLER LIKE' 
2 ' LIHOOD) • I 

418 WRITE! 6,207) (XPI(I),I=l»M) 

207 FORMAT! 8X, 'PI : ' 2X , 12 ( F4. 2,2X1 ) 

IF(KPI414,414,261 
261 WRI TE ( 6 ,416 1 
GO TO 419 

414 WRI TE ( 6,4071 

407 F0RMAT(1H+,88X, 'OBTAINS'/) 

WRI TE ( 6,2061 

206 F0RMAT(2X, 'SUM OF • 22X ,' NUMBER OF SUCCESS ES • 40X ,' CUM • 
l'ULATI VE »5X, 'CUMULATIVE •/, IX, 'SUCCESS ES'79X, 'PROBAB* 
2* ILITY *4X, 'PROBABILITY'/, 89X, 'UNDER H0'7X, 'UNDER HI' 
3//I 

RECOVER THE VECTORS ( K1 ,K2,. . . , K12 I AS (LL < 1 1 ,LL ( 2 1 , 
• « . , LL ( 12 1 I STARTING WITH THE ONE WITH GREATEST 
LIKELIHOOD RATIO, THUS FORMING THE ACCEPTANCE 
REGION 

419 DO 170 L=1 , N 
KU=KX(L» -1 
MB=13-M 

DO 160 I =MB ,12 
K=1 3- I 

LL ( K ) =MOD ( KU , ML ( K I I 
KU=KU/ML(K) 

160 CONTINUE 

COMPUTE THE PROBABILITY MASS FOR THE VECTOR 
< LL ( 1 1 ,LL(2) ,...LL(12) I UNDER HO AND HI 
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U = l. 0 

DO 165 1=1, M 
420 KL=LL( I »+l 

U=U*PA!ML! I > , K L , I > 

165 CONTINUE 

W=XLAM< L ) *U 

DECREMENT CUMULATIVE DISTRIBUTION FUNCTIONS 

V=F 
F=F-W 
VV=Q 
0=0-U 
L Y = l 

RECOVER SUM OF COMPONENTS OF VECTOR 

DO 168 1=1,12 
LY=LY+ LL ( I I 
168 CONTINUE 

TAG THE VECTOR WHOSE ENCODED VALUE IS L WITH THE 
VALUE OF THE SU* OF ITS COMPONENTS 

K SUM ( L > =L Y 

C CUNT THE NUMBER OF VECTORS SO FAR ACCOUNTED FOR 
WHOSE SUM IS LY 

KFY!LY>=KFY!LY)+1 
IF< KP) 262 ,262,263 

START PRINTING THE DISTRIBUTIONS WHEN THE SIZE 
VALUE REACHES START 

262 IF( START-FI263 ,264,264 
264 KZ=LY-1 

WRI TE ( 6 , 210 I KZ 
210 FORMAT ( 4X , I 4 ) 

WRITE (6 .4081 <LL!I>,I=1,M) 

408 FORMAT! 1H+ , 14X ,12( 12, 4X) ) 

WRI TE(6,409)V,VV 

409 F0RMAT(1H+,88X,2(F7.5 ,8X) ,//» 

STOP PRODUCING ADDITIONAL VECTORS WHEN THE DESIRED 
SIZE VALUE IS REACHED 

263 IF! SIZE-F) 170,170,415 

170 CONTINUE 

IF CALLED FOR, PRINT SUMMARY 

415 IF(KP)267,171,171 

171 WRITE<6,225) V 

225 FORMAT ( 5X , ‘THE FOLLOWING IS A SUMMARIZED STATEMENT • 
1 • OF THE REJECTION AND ACCEPTANCE REGIONS FOR A TEST* 
2* OF SIZE»F6.3,*. • / / > 

FOR THOSE VECTORS WHOSE SUM OF COMPONENTS IS I AND 
WHICH WERE ALL IN THE ACCEPTANCE REGION ASSIGN 
KT ( I )=2 TO SO DENOTE 

DO 180 1=1, KYT 
NY! I ) =NY ( I »-KF Y ( I ) 

I F ( NY ( I ) 1181 ,181,182 
181 KT! I)=2 
GO TO 180 

FOR THOSE VECTORS WHOSE SUM OF ELEMENTS IS I AND 
WHICH WERE ONLY PARTIALLY ACCOUNTED FOR IN THE 
ACCEPTANCE REGION ASSIGN KT ( I I = 1 TO DENOTE THAT 
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A MORE OETAILED INVESTIGATION IS REQUIRED. FOR THOSE 
VECTORS WHOSE ELEMENTS SUM TO I AND FOR WHICH NONE 
APPEAR IN THE ACCEPTANCE REGION SET KT(I»=0. 

182 IF(KFY( I )) 183,183*184 

183 KT( I )=0 
GO TO 180 

184 KT( I > = 1 
180 CONTINUE 

WRI TE ( 6 »23G ) V 

230 FORMAT ( 6X, • ALL VECTORS WHOSE COMPONENTS SUM TO THE * 

1 'FOLLOWI NG NUMBERS ARE IN THE ACCEPTANCE REGION FOR* 

2* A TEST OF SI ZE • F6.3 «••*// ) 

PRINT THOSE VECTORS OF THE ACCEPTANCE REGION WHICH 
CAN BE IDENTIFIED SIMPLY BY THE SUM OF COMPONENTS. 

DO 190 1=1, KYT 
K = I — 1 

IF! KT (I)-l)191, 191*192 
192 WRI TE ( 6,231 > K 

231 FORMAT(25X,I3,//l 
KJ=I+1 

190 CONTINUE 

INVESTIGATE VECTORS FCR WHICH KT!I»=1 AND PRINT 
THOSE WHICH ARE IN THE ACCEPTANCE REGION. 

191 IF(KYT+1-KJ)267,267,266 

266 DO 195 I =K J » KYT 

I F ( KT ( I ) 1197,197,194 

194 K I = I 

195 CONTINUE 

197 IF(KI-KJ)120,196,196 

196 WRI TE ( 6 ,232 ) V 

232 FORMAT ( 6X » ' THE ACCEPTANCE REGION FOR A TEST OF SIZE* 
1F6.3,1X, 'ALSO INCLUDES THE FOLLOWING VECTORS: •//, 7X, 

2* SUM OF ' Z,5X, • COMPONENTS • 30 X , ’VECTOR'//) 

DO 110 I =K J , KI 
DO 110 J=1,L 

I F ( K SUM C J I — I ) 110,111,110 

111 KU=KX(J)-1 

DO 112 K=MB ,12 
KK=13-K 

LL( KK|=MOD(KU,ML( KK ) ) 

KU=KU/ML( KK» 

112 CONTINUE 
KZ=KSUM( J ) -1 
WRITE(6,237»KZ 

237 FORMAT! 10X, 12 ) 

WRI TE( 6,233) (LL(K),K=1.M) 

233 FORMAT! 1H+,25X,10( 12, 4XH 
WRITE! 6,236) 

236 FORMAT (IX,/) 

110 CONTINUE 

PRINT THE CONCLUDING STATEMENTS 
120 WRI TE ( 6,2341 V 

234 FORMAT ( 6X. • ALL GTHER VECTORS ARE IN THE REJECTION R* 

1 * EG I ON FOR A TEST OF SI ZE* F6. 3, * . • // I 

WRITE(6,235» VV.V 

235 F0RMATI6X, *THI S TEST HAS POWER OF* F6. 3, IX, » AT SIZE*F6. 

267 CONTINUE 
END 
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